Skip to content

ADR-0001: Expose the Rust orchestration engine over HTTP and reduce the Cloud Function to a proxy - #2

Closed
nicholas-ruest wants to merge 5 commits into
adr/0002-certification-document-integrityfrom
adr/0001-implement-real-orchestration-logic
Closed

ADR-0001: Expose the Rust orchestration engine over HTTP and reduce the Cloud Function to a proxy#2
nicholas-ruest wants to merge 5 commits into
adr/0002-certification-document-integrityfrom
adr/0001-implement-real-orchestration-logic

Conversation

@nicholas-ruest

Copy link
Copy Markdown
Member

Implements docs/adr/ADR-0001-implement-real-orchestration-logic.md.

⚠️ Stacked on #1. Base branch is adr/0002-certification-document-integrity, not main. ADR-0001 step 1 is a hard gate on ADR-0002's three test fixes, so branching from main would have meant claiming a gate was satisfied when it was not. Retarget to main once #1 merges. The diff shown here is this ADR's work only.

What changed

The deployed service now orchestrates. Before this, every handler in functions/index.js was a synchronous pure function with no I/O that echoed tasks.length back with a status field. A cyclic workflow returned status: "resolved" with HTTP 200. Nothing ever called the Rust engine.

Seven routes added to the axum server (crates/llm-orchestrator-cli/src/orchestrator_routes.rs, new module, mounted at main.rs:4670). Each deserializes into the agent's existing request type and awaits the existing async method. Nothing in llm-orchestrator-core changed — the routes were the missing link, exactly as the ADR predicted.

Route Agent Method
/v1/orchestrator/workflow WorkflowOrchestratorAgent execute
/v1/orchestrator/scheduler TaskSchedulerAgent schedule
/v1/orchestrator/dependencies DependencyResolverAgent resolve
/v1/orchestrator/retry RetryRecoveryAgent evaluate
/v1/orchestrator/parallel ParallelizationAgent analyze
/v1/orchestrator/state-machine StateMachineAgent transition
/v1/orchestrator/swarm SwarmCoordinatorAgent coordinate

Envelope ported byte-compatibly. execution_metadata (trace_id, timestamp, service, execution_id) and layers_executed are rebuilt in Rust to match functions/index.js:53-74, including the ORCHESTRATOR_<AGENT> layer naming and its hyphen→underscore substitution. A test pins each field against the JavaScript it replaces.

Validation and error mapping ported. Required-field checks reproduce validateRequiredFields including its exact Missing required fields: a, b message. Cycles, dangling dependencies and malformed bodies are 400; engine faults are 500.

The Cloud Function is now a proxy. It keeps CORS, routing and contract publishing. It forwards each agent request to ORCHESTRATOR_ENGINE_URL with a Google-signed ID token fetched from the metadata server (no new dependency — Node 20 fetch), propagates X-Correlation-ID, and returns the upstream status and body unmodified. The local state-machine transition table at :191-193 is deleted, so there is one source of truth. Dispatch now awaits.

Everything fails closed: unconfigured → 503, unreachable → 502, timeout → 504, non-JSON upstream body → 502. /health and /ready reflect the engine rather than checking that seven constants are seven constants.

Real test output

Rust route testscargo test -p llm-orchestrator-cli --bin llm-orchestrator:

running 13 tests
...
test result: ok. 13 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

These include the ADR's own Verification section, run against the real agents:

ADR test Test name Result
Test 1 — diamond DAG in topological order, B and C in one parallel group diamond_dag_comes_back_in_dependency_order
Test 2 — cycle rejected with 400 cycle_is_rejected_with_400
Test 3 — dangling dependency rejected with 400 dangling_dependency_is_rejected_with_400

Test 1 posts the four tasks in an order that is not a valid execution order, so echoing the input cannot pass. Test 2 gets HTTP 400 where the old handler returned 200 with status: "resolved".

Node proxy testsnpm test:

========================================
Results: 113 passed, 0 failed, 113 total
========================================

Including ADR Test 5 (readiness fails closed): /ready returns 503 with ready: false when the engine is unreachable, where it previously returned 200 unconditionally.

Full workspacecargo nextest run --all --lib --bins --no-fail-fast:

Before this PR After
passed 432 445
failed 3 3
skipped 2 2
total 437 450
Summary [14.119s] 448 tests run: 445 passed, 3 failed, 2 skipped

No regressions. llm-orchestrator-core stays 198/198. The +13 are exactly the new route tests. The 3 failures are the same pre-existing ones documented in #1 (two cohere_embeddings mockito aborts, one timing-sensitive secrets cache test) — untouched by this work. docs/TEST_CERTIFICATION.md is regenerated from this run.

Deviations, and why

1. /v1/orchestrator/workflow does not accept the contract's flat shape. The published contract requires { workflow_id, workflow_name, tasks }, but ExecuteRequest is { workflow: Workflow, inputs } and Workflow.steps needs a StepType and a step config per step. The contract's tasks carry neither. I did not invent a mapping — a guessed translation would execute something the caller did not ask for, on an endpoint that runs real LLM steps. The route returns 400 naming the shape it wants. Reconciling the contract with ExecuteRequest is a contract change and needs a human decision.

2. Three request types require a timestamp that no contract lists as required. StateTransitionRequest, ParallelizationRequest and SwarmCoordinationRequest all declare timestamp: DateTime<Utc> with no serde default, so a contract-conformant body fails to deserialize. Fixing it properly means changing agentics-contracts, which edge-agent and inference-gateway each vendor and which the ADR puts explicitly out of scope. The routes fill the value in at the boundary instead. This is a latent contract bug worth a follow-up.

3. The retry contract's failure field is error in Rust. RecoveryRequest names it error: FailureDetails. The route accepts the contract's failure name on the wire and renames it, rather than breaking existing callers.

4. The agent is constructed per request. StateMachineAgent::transition takes &mut self and keeps history in memory. Sharing one across requests would leak one caller's entity states into another's validation, so each request gets a fresh agent — the same thing the CLI does, and the same persistence behaviour as the handler it replaces (none).

⚠️ Left undone — needs a human

Steps 6 and 7: deploy to Cloud Run and grant IAM. I updated deploy/gcloud/setup-iam.sh to grant the Cloud Function's service account roles/run.invoker on the llm-orchestrator service specifically rather than project-wide, and it degrades gracefully if the service does not exist yet. I did not run it, and did not deploy. Deploying to agentics-dev is a production change requiring GCP credentials and a human decision about rollout order. The sequence a human needs:

  1. ./deploy/gcloud/deploy.sh agentics-dev dev us-central1 — builds and deploys the engine.
  2. ./deploy/gcloud/setup-iam.sh agentics-dev dev us-central1 — grants the function invoker access (set FUNCTION_SA_EMAIL if the function does not use the default App Engine SA).
  3. Set ORCHESTRATOR_ENGINE_URL on the Cloud Function to the Cloud Run URL. Until this is set the function returns 503 for every agent route — deliberately, since failing closed is the point.
  4. Re-run the ADR's Verification tests against the deployed pair.

Rollout risk to check first: responses change from always-200-with-placeholder to genuinely failing on invalid input. Any downstream consumer that assumes success will start seeing 400s. The ADR calls this out and it has not been audited.

Step 12: the CI job that runs Verification against the container. Blocked by the same missing workflow token permission described in #1.github/workflows/ci.yml cannot be pushed from here.

Latency SLOs. Any latency figure in docs/ derived from the old stub timings measured JSON echoing and is now invalid. Not re-measured.

ADR Verification criteria

# Criterion Status
1 Diamond DAG returns correct topological order; B and C share a parallel group diamond_dag_comes_back_in_dependency_order
2 Cycle rejected with 400 cycle_is_rejected_with_400
3 Dangling dependency rejected with 400 dangling_dependency_is_rejected_with_400
4 Ordered execution actually happens (2-step workflow, stub provider) not done — needs a stub provider and the contract reconciliation in deviation 1
5 Readiness fails closed ✅ Node suite, engine unreachable → 503 / ready: false
Regression gate: cargo test --all green ❌ pre-existing agentics-contracts::integration_tests breakage, see #1

target/ was removed after building; the working tree is clean.

…on to a proxy

The deployed service now orchestrates. Before this, `functions/index.js` answered every agent
request with a synchronous pure function that echoed `tasks.length` back with a status field:
a cyclic workflow returned `status: "resolved"` with HTTP 200, and no handler ever called the
44,189 lines of tested Rust that implement the real thing.

Adds the seven `/v1/orchestrator/*` routes to the existing axum server in a new
`orchestrator_routes` module, each deserializing into the agent's existing request type and
awaiting the existing async method. Nothing in llm-orchestrator-core changed; the routes are
the missing link the ADR identified.

Details worth knowing:

- The response envelope (`execution_metadata`, `layers_executed`) is rebuilt in Rust to be
  byte-compatible with `functions/index.js:53-74`, including the `ORCHESTRATOR_<AGENT>` layer
  naming and its underscore substitution, so existing consumers see an unchanged shape. A test
  pins each field against the JavaScript it replaces.
- Required-field validation reproduces `validateRequiredFields` including its exact
  `Missing required fields: ...` message.
- Errors map by cause: a cycle, a dangling dependency, or a bad request body is 400; engine
  faults are 500. The dependency resolver reports a cycle in its status rather than as an Err,
  so an unsuccessful resolution is explicitly mapped to 400 -- returning 200 there would
  reproduce the exact defect this removes.
- `StateTransitionRequest`, `ParallelizationRequest` and `SwarmCoordinationRequest` all require
  a `timestamp` that none of the published contracts list as required, so a contract-conformant
  body would fail to deserialize. The routes fill it in at the boundary; reconciling the two is
  a change to agentics-contracts, which two other repositories vendor and which ADR-0001 puts
  out of scope.
- `/v1/orchestrator/workflow` takes the engine-native `{ workflow, inputs }` shape and rejects
  the contract's flat `{ workflow_id, workflow_name, tasks }` shape with a pointer to the right
  one. Those `tasks` carry no step type or configuration, so there is no faithful translation
  into `Workflow.steps` and guessing one would execute something the caller did not ask for.

The Cloud Function keeps CORS, routing and contract publishing and stops fabricating results.
It forwards each agent request to `ORCHESTRATOR_ENGINE_URL` with a Google-signed ID token
fetched from the metadata server, propagates `X-Correlation-ID`, and returns the upstream
status and body unmodified. Every failure mode fails closed -- unconfigured 503, unreachable
502, timeout 504, non-JSON upstream body 502 -- and `/health` and `/ready` now reflect the
engine instead of checking that seven constants are seven constants. The local state-machine
transition table is deleted so there is one source of truth.

`functions/test.js` is rewritten to assert proxy behaviour instead of envelope shape: the old
suite asserted only shape and so passed against a service that orchestrated nothing.

Also drops the unused `claude-flow` dependency, and grants the Cloud Function's service account
`roles/run.invoker` on the engine service specifically rather than project-wide.

Implements orchestrator/docs/adr/ADR-0001-implement-real-orchestration-logic.md.
445 passed, 3 failed, 2 skipped of 450, up from 437 -- the 13 new tests are the
orchestrator_routes suite. llm-orchestrator-core remains 198/198 and the three failures are the
same pre-existing ones ADR-0002 surfaced, unchanged by this work.

Implements orchestrator/docs/adr/ADR-0001-implement-real-orchestration-logic.md.
ADR-0001 is stacked on ADR-0002, so the certification job that runs the Rust route tests lands
here via its base branch rather than being duplicated.
…n returning

The Cloud Function's tests had no CI job at all -- before this branch the suite asserted only
envelope shape, and even that never ran. Now that the function is a proxy, its 113 tests assert
the properties that matter: every agent request reaches the engine on the right path with the
correlation ID attached, the engine's status and body come back unmodified, and every failure
mode fails closed (unconfigured 503, unreachable 502, timeout 504, non-JSON upstream 502,
readiness non-200). No install step is needed: no lockfile is committed and the suite stubs
fetch rather than reaching the network.

A second step greps for the two constructs that would mean the function had started answering
by itself again -- the `tasks_count: Array.isArray(...)` echo and the local state-machine
transition-table lookup. Both were verified to fire against the version of functions/index.js on
main, so they are not vacuous checks.

The Rust route tests, including ADR-0001's diamond-DAG, cycle and dangling-dependency
verification tests, need no new job: they are bin-target tests and already run in the nextest
step this branch inherits from ADR-0002. They appear in the certification as
`llm-orchestrator-cli | 13 | 0 | 0 | 13`.

Not done, and not fakeable here: step 12's containerised end-to-end job. `serve_http` calls
`validate_phase3_startup`, which requires RUVECTOR_SERVICE_ENDPOINT and RUVECTOR_API_KEY and
hard-exits when the Ruvector service is unreachable, deliberately, to trigger a Cloud Run
crashloop on misconfiguration. A CI container would therefore crashloop, and standing up a fake
Ruvector is new infrastructure ADR-0001 does not specify. Recorded in the pull request instead
of guessed at.

Implements step 12 of orchestrator/docs/adr/ADR-0001-implement-real-orchestration-logic.md, in
part.
The Lint job runs `cargo fmt --all -- --check`, and the new module had 5 diffs. Formatted with
rustfmt directly rather than `cargo fmt -p` so main.rs's 38 pre-existing diffs are not swept into
this branch as unrelated churn.

The job stays red regardless: the repository has 521 formatting diffs on main and this branch
does not take that on. Clippy reports nothing against the new module.
@nicholas-ruest
nicholas-ruest deleted the branch adr/0002-certification-document-integrity July 27, 2026 20:27
@nicholas-ruest
nicholas-ruest deleted the adr/0001-implement-real-orchestration-logic branch July 27, 2026 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant